Approximation of functions by perceptron networks with bounded number of hidden units

We examine the e ect of constraining the number of hidden units For one hidden layer networks with fairly general type of units including perceptrons with any bounded activation function and radial basis function units we show that when also the size of parameters is bounded the best approximation property is satis ed which means that there always exists a parameterization achieving the global minimum of any error function generated by a supremum or Lp norm We also show that the only functions that can be approximated with arbitrary accuracy by increasing parameters in networks with a xed number of Heaviside perceptrons are functions equal almost everywhere to functions that can be exactly computed by such networks We give a necessary condition on values that such piecewise constant functions must achieve